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Abstract 



This paper presents research on middle school students working on large science projects in the 
Knowledge Integration Environment. The research indicates that metacognitive scaffolding 
provided through prompts improves the quality and quantity of students' work. The two studies 
discussed here investigate students' use of two types of metacognitive prompts: activity-focused 
prom.pts and self-monitoring prompts. Activity prompts walk students through the important 
steps in the activities comprising a large science project (such as providing justifications or 
explaining decisions), while self-monitoring prompts encourage planning for and reflection on 
those activities. Activity prompts are found to help students complete all the pieces of a project, 
but self-monitoring prompts encourage students to demonstrate an integrated understanding of 
the relevant science. Possible cognitive mechanisms are proposed. 



Introduction 

The objective of this research is to develop an exemplary program of metacognitive 
prompts to encourage principled knowledge integration by students working on 
large science projects. Through this research, I address two questions. First, 
what effect do "self-monitoring" and "activity-focused" metacognitive prompts 
have on middle school students' ability and propensity to develop an integrated 
understanding of science topics? Second, what is the explanation for the 
difference in effects of the two types? 

Two studies address these questions. The first section of this paper presents a 
description of the relevant aspects of the Knowledge Integration Environment, the 
Computer as Learning Partner classroom, and the theoretical framework on 
which the research is based. The second section of the paper presents the 
methods used in the two studies, and the third section summarizes the findings. 
In the final section, I discuss the contributions of the research to educational 
practice and theor}^ 

Context for the Research 

The Knowledge Integration Environment (KIE) implements projects centered 
around understanding scientific evidence on the World-Wide-Web. KIE blends 
custom and commercially- available software. The KIE software is used by 
students participating in curriculum units developed by the KIE research group 
and others. Those units, called "projects", are designed to encourage students to 
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attain a deep understanding of science concepts rather than to collect scientific 
facts, and have been used successfully as a venue in which students can apply the 
science principles they have been learning in class. KIE projects fall in three 
major categories: critiques, debates, and design projects. Critique projects foster 
the development of a critical eye when using evidence and evaluating arguments. 
Debates help students see that multiple sides may exist to arguments and that 
evidence should be used effectively to improve those arguments. Design projects 
engage students in an application of their knowledge. 

Among other sites, KIE is being implemented in the Computer as Learning 
Partner (CLP) classroom. CLP provides a one-semester curriculum introducing 
the physical science topics of thermod 3 naamics, light, and sound to eighth-graders. 
In the curriculum, which is laboratory-based, the computer is used as a tool to 
collect and graph real-time data, perform simulations of experiments that 
consume too much time to do during a class period, and help students track their 
progress in moving through the sections of the lab. The KIE software and 
curricula have developed directly out of CLP's decade of research on designing 
curricula and technology for middle school science teaching and learning. The 
KIE projects discussed here have been designed to complement the CLP 
curriculum. 

KIE and CLP are guided by a framework called "scaffolded knowledge 
integration." The scaffolded knowledge integiation instructional framework has 
four elements (Linn, 1995). First, we make thinking visible to students by 
illustrating diverse forms of expertise and reasoning about scientific phenomena. 
Second, we identify accessible models for scientific phenomena to help students 
connect new information to existing knowledge and to problems that are both 
familiar and relevant. Third, we provide social supports for the students as they 
learn. Finally, we engage students as lifelong and autonomous learners — as both 
investigators and critics — so they can recognize the various sides of arguments, 
identify weaknesses in their own and others' arguments, and identify ways to 
strengthen those arguments. 

Prompts represent one aspect of the scaffolding in KIE. The research discussed 
in this paper focuses in particular on "activity prompts" and "self-monitoring 
prompts", which provide metacognitive opportunities for students. Activity 
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prompts serve as memory support, while self-monitoring prompts promote 
reflection. The following section will discuss the rationale for such prompts. 

Reflection through metacognitive prompts 

This research on prompts has grown out the extensive history of the idea of 
"reflection." Today's educational researchers generally accept the need for 
reflection in students' learning, although it is not always incorporated into 
instructional practice. Reflection provides one method for fostering conceptual 
change and knowledge integration by helping students to differentiate among 
ideas and make connections between them. Reflection can be facilitated through 
many different mechanisms and can focus on different aspects of the learning 
process. Here, the investigation is restricted to prompting as a mechanism for 
reflection. In particular, I investigate aspects of students' planning and 
monitoring of their learning activities. 

One way reflection has been encouraged in the classroom has been through 
teacher questioning. In recent decades, though, there has been increased interest 
in the types of questions being asked and answered. In the 1950s, the inquiry 
teaching method was popular. At that time, teachers were taught that they must 
ask students questions and get them to be active participants in their own 
education. Bloom's taxonomy characterized the nature of questions, and 
determined, distressingly, that teachers asked primarily fact-oriented questions. 
As a result, teachers were then encouraged not only to ask more questions, but to 
make those questions of a higher order than they had previously been, since 
asking higher order questions is thought to get students to continue the process of 
reflection on their own. Programmed instruction provided one attempt to have 
students answer these important questions. 

Years later, collaborative learning has provided a new way of addressing these 
issues. Recent research seeks to foster the development in students of self- 
sustaining thinking skills through the use of these higher order questions. For 
example, in reciprocal teaching, used most often in reading classes, students 
prompt each other to explain using a set of general questions that can be asked 
about any kind of paragraph (Brown & Palincsar, 1989). Situated cognition and 
cognitive apprenticeship emphasize the importance of t 3 dng the questions being 
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asked to the subject matter at hand (Brown, Collins, & Duguid, 1989; Collins, 

Brown, & Newman, 1989). 

Many researchers have investigated the importance of explanations in 
understanding. Students who self-explain demonstrate superior problem solving 
ability, and research indicates that eliciting self-explanations leads to improved 
understanding of texts (Chi, Bassok, Lewis, Reimann, & Glaser, 1989; Chi, 
deLeeuw, Chiu, & LaVancher, 1994; see also Webb, 1983). Bielaczyc (Bielaczyc, 
Pirolli, & Brown, 1995) found that students can be trained to give self-explanations, 
and that this training also facilitates the end result of improved understanding of 
the procedural domains in which the research has been performed. Bielaczyc s 
study provided the first evidence of the existence of a causal (rather than 
correlative) relationship between giving self-explanations and improved ability, in 
this case in computer programming. 

We have reason to believe that giving explanations benefits students' deep 
understanding, as well. Linn (1995) notes that knowledge integration in science 
comes about when students link and connect ideas, leading them to develop a 
cohesive view' of science and to be able to apply their knowledge of science to new, 
personally-relevant problems. KIE projects are designed to facilitate development 
of this type of deep understanding, and prompts serve as part of that facilitation. 
The explanations investigated here, then, will be written ones made in response to 
prompts. KIE's prompts are similar to those used in CSILE (Scardamalia & 
Bereiter, 1991). From a Vygotskian perspective (Vygotsky, 1934/1962), prompts 
help students move into their zone of proximal development: prompts provide an 
opportunity for students to make self-explanations, and give students a better 
understanding of the kinds of questions they should be addressing. Scardamalia 
and Bereiter (1991; Bereiter & Scardamalia, 1987) have found that, with, proper 
external supports to focus their attention on the salient planning and justification 
issues of writing, middle school students can "transform" their knowledge into 
more than a series of facts or statements rather than relying on a knowledge 
telling strategy for writing. They also find that prompted students show increased 
metacognitive activity. 

The prompts in KIE focusing on explanations are activity-specific; that is, they are 
developed with a particular activity, such as critiquing evidence, in mind. 
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Research indicates that this specificity is important. Linn and Clancy (1992j have 
found that students are most successful at learning when they are working 
within a concrete context rather than at an abstract level. Davis (1995) found that 
students benefit from contextualized prompts that help them clarify and focus 
their thinking. Linn and Songer (1991) found that prompting students to make 
predictions and obseiwations in labs encourages integration of ideas. And, in 
reciprocal teaching, students essentially internalize an appropriate program of 
prompts for a particular activity — for example, reading a passage (Brown & 
Palincsar, 1989). All of these research efforts point to the importance of providing 
contextualized scaffolding rather than asking abstract questions. Similarly, 
activity prompts in KIE are designed to help students to identify appropriate, 
detailed considerations as they work on individual activities within the context of 
a large project. This guidance helps students to convince others (e.g., their peers 
and teachers) that their work is valid, by helping them to work through aspects of 
the expert thinking process. An activity prompt might, for example, provide 
students with the opportunity to justify their decisions or write scientific 
explanations of those decisions. For some students, activity prompts may seiwe 
simply as reminders. For others, they may act as pointers to new considerations. 

Prompts to encourage self-monitoring are also beneficial. White and Frederiksen 
(1995) have shown great success with planning and reflection prompts with self- 
assessment activities in their mechanics curriculum for junior high students. 
Students who routinely answered such prompts developed greater understanding 
of the subject matter. Some of the reciprocal teaching prompts, for example those 
that encourage students to assess their own undert^tanding of the material, also 
address monitoring issues. Likewise, KIE's self-monitoring prompts are 
planning and reflection prompts designed to help students map out their 
strategies for an activity, and then reflect back on that activity and identify their 
work's strengths and weaknesses. Self-monitoring prompts encourage students 
to think carefully about their own activities, encouraging planning and reflection, 
rather than focusing only on the short-term, smaller goals. The prompts help 
students think about large problems in ways an expert might consider 
appropriate: considering the overarching goal or goals, identifying ways in which 
to accomplish those goals, evaluating one's progress at regular inteiwals, and 
developing strategies for improving one's understanding of the problem at hand. 
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Thus, properly constructed programs of prompts can serve two roles in achieving 
the overarching goal of knowledge integration: they provide the impetus for 
explanation (and thus facilitate the process of knowledge integration in the 
particular context in which they are given), and they encourage reflection at a 
level than students do not generally consider (and thus help students develop the 
thinking strategies they need for effective knowledge integration and lifelong 
learning). 



Methods 

This research undertakes to measure the relative success of different types of 
prompts and focuses on developing prompts that are useful to different kinds of 
learners. Two studies have been conducted already. (A third is currently 
underway.) The methods used for the two studies will be discussed here, after a 
discussion of the development of the prompts in KIE. 

Design of Prompting 

Scaffolding is present in various forms throughout the KIE software and 
curriculum. Scaffolding is also called "guidance", a more student-friendly tenn. 
(Figure 1 provides a diagrammatic representation of the scaffolding explicitly 
provided by KIE.) The word processing files with which the students work provide 
explicit, interactive, bi-directional guidance, called "prompts". Prompts appear in 
the form of sentence-starters and students write responses that complete the 
sentences. The software also provides uni-directional guidance, in the form of a 
checklist and hints from the software's cognitive guidance system, called 
"Mildred." Students do not make written responses to this type of scaffolding. The 
checklist helps students keep track of what they are doing, and the hints help 
them with both logistical and conceptual issues. 
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Bi-directional 
Guidance 
y (Prompts) 

Scaffolding 

(Guidance) 

Uni-directional 
Guidance 



Activity Prompts 



V 



Self-Monitoring Prompts 
Content Prompts 

Logistical 

\ 



Hints 



Checklist 



Conceptual 



Figxire 1; Scaffolding in KIE 



The guidance provided by KIE builds on our experiences with CLP. For example, 
sentence-starter prompts have been used successfully in the CLP laboratory 
software. With the help of these prompts, students learn to make predictions, 
analyze results, and draw conclusions in their lab "reports." The CLP software 
also provides a checklist of activities. The checklist significantly reduces the 
amount of time the teacher spends answering procedural questions (e.g., "What 
do we do next?") and frees up time for conceptual questions (e.g., "Why would the 
temperature drop along a curve like that?"). The CLP software also provides 
logistical help with using the software, and provides cognitive guidance to help 
students think about each activity. 



Procedural, logistical, and conceptual guidance are available, then, in both the 
CLP and KIE software. Although the presentation is different, the intent is 
largely the same. The primary difference in the two approaches is that KIE, 
because of its potential use by students working without close contact with a 
teacher, has a greater emphasis on conceptual guidance in the form of both hints 
and prompts. Activity prompts and self-monitoring prompts are two types of that 
conceptual guidance from which students benefit. 



Methods for Study 1 

The first study of the metacognitive prompts was performed in the context of the 
design project called "Aliens On Tour." The goal of the study was to assess the 
effect of self-monitoring prompts, which were additional requirements for half of 
the students. For the "Aliens" project, students were asked to design houses and 
clothing for three sets of cold-bloodeu aliens with different climate requirements. 
The evidence the students reviewed included data on insulation properties of 
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different materials, advertisements for clothing items or "coolers," and other 
relevant information. The students were nearing completion of the semester-long 
CLP course focusing on heat and light energy, and were expected to bring their 
knowledge of these areas into their designs, as well. Students worked in pairs or 
occasionally triads, and they created 65 house and clothing designs altogether. 

The students reviewed the evidence, then created their designs consisting of text 
and, in some cases, graphics. KIE provided general instructions as to what 
information should be included in the designs, as well as an admonition to back 
up all decisions with scientific evidence. As they completed each design, students 
saw an activity prompt asking why their design would work well for the different 
types of aliens. This prompt, along with the general instructions given, helped 
students think about the justification necessary to demonstrate to others the 
quality of their design, a process quite familiar to expert scientists. The activity 
prompt stated, "Our design will work well because...." 

In addition, half of the students (three class periods) received seven specific self- 
rnonitoring prompts at the beginning and end of each activity of the project (in 
other words, before and after they reviewed the evidence, and before and after each 
of the two designs). The "plan ahead" prompts asked them to write about what 
they needed to think about for the activity. A plan ahead prompt used in the 
design activity was, "In thinking about doing our design, we need to think 
about...." The "look back" prompts asked them to reflect on what aspects of the 
activity were still confusing to them, or how their designs could be better. A look 
back prompt used was, "Our design could be better if we...." Appendix A provides 
a subset of the documents with which students in this condition worked. 

Thus, in "Aliens", two conditions were investigated; one group received activity 
prompts, and the other received self- monitoring prompts in addition to the activity 
prompts. Table 1 summarizes the conditions in this study and in Study 2. 
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Period 


Study 1 
(Aliens, S95) 


Study 2 
(ATN, F95) 


1 


AP 


AP 


2 


teacher prep. 


AP 


3 


SMP + AP 


BP (Control) 


4 


SMP + AP 


SMP 


5 


SMP + AP 


teacher prep. 


6 


AP 


teacher prep. 


7 


AP 


SMP 



Table 1: Studies 1 and 2* 

* AP = Activity Prompt, SMP = Self-Monitoring Prompt, BP = Beliefs 
Prompt; teacher prep. = no class meeting 



Students' specific responses to the two types of prompts were coded according to 
their content. The coding scheme for the self-monitoring prompts noted the 
aspects on which the students chose to focus (for example, a goal restatement or a 
science question). For the activity prompts, the coding scheme identified whether 
the responses contained scientific content or not. Students' final designs were 
also coded. The salient aspects of the project coding were: level of explanation 
(scientific, explanatory, or descriptive), number of scientific principles used, and 
overall solution quality. All these results were incorporated into chi-square tests 
to compare the two conditions. 

Methods for Study 2 

Study 2 was undertaken for three purposes: (1) to test similar (but refined) 
prompts in a different project, with the intent of comparing students' responses in 
different contexts; (2) to investigate separately the activity and self-monitoring 
prompts to identify their individual strengths and weaknesses; and (3) to create a 
program of each type of prompts that would allow students in each condition to 
spend comparable amounts of time on the actual project work itself. 

Studies 1 and 2 took place in sequential semesters. Study 2 was informed by the 
results of the first. In the critique project called "All The News", students 
critiqued evidence cited by a news article about energy conversion. The project 
involved (a) reading the article to be critiqued and looking at its concomitant 
evidence (about heat flow and energy conversion), (b) critiquing the evidence and 
the claims being made, and (c) writing a letter to the imaginary "editor" 
explaining the critique made and giving guidelines for future use of evidence. 
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"All The News", unlike "Aliens", took place early in the semester. Again, 
students worked in pairs or triads on the project. 

In Study 2, the activity prompts underwent significant refinement. The new 
activity prompts grew out of those in "Aliens", but included not just prompts for 
justification but for all the steps necessary to do a good job on the project. For 
example, students were prompted for the guidelines for using evidence with the 
prompt, "When you think about evidence, in general you should make sure to 
consider...." Appendix B provides some of the documents with which students in 
this condition worked. 

The self-monitoring prompts were also refined in "All The News." The basic 
refinement made to the self-monitoring prompts was to include thinking types 
before each prompt, similar in nature to those used in the CSILE program 
(Scardamalia & Bereiter, 1991), to cue the students as to what kinds of planning 
and reflection are important. Examples of thinking types include "Thinking 
Ahead", "WTiat to Include", "Checking Our Understanding", and "How We Spent 
Our Time." One self-monitoring prompt for planning the "letter to the editor" 
was, "Specific things we need to think about as we write our letter include..."; one 
reflection self-monitoring prompt from that activity was "In looking back at what 
the editor wanted, we think she will like our letter because...." Appendix C 
provides a subset of the documents with which students in this condition worked. 

A set of "belief prompts" were used in the control condition to ensure that all 
students spent similar amounts on the project itself. These prompts asked 
students about their beliefs about using and critiquing evidence. An example 
prompt is, "If two people believe two different claims, it means...." Belief prompt. 
acted as time fillers and were unlikely to influence students' learning. 

In Study 2, then, three conditions were investigated: two periods received activity 
prompts (the AP condition), two periods received self-monitoring prompts (the 
SMP condition), and one period received beliefs prompts (the BP, or control, 
condition). See Table 1 for a summary of the conditions. 

Similar coding schemes were used for Study 2 as for Study 1. The self-monitoring 
prompts coding was refined to allow identification of major themes across 
multiple prompt responses. These qualitative themes, though , provide insight 
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into what students are thinking about as they work with self-monitoring prompts. 
The major refinement to the first study's analysis was to the project coding, which 
for Study 2 focused on (a) principled knowledge integration as demonstrated by the 
"cites" students gave for their explanations, (b) the level of explanations given, and 
(c) the characteristics of the guidelines they provided. The analysis made use of 
chi-square tests and Fisher Exact tests. 

Results: Measuring the Success of Prompts 

To assess the success of the metacognitive prompts, we can investigate (1) the 
degree to which the students complete the pieces of the project required, and (2) 
the quality of those pieces. The following sections discuss the findings regarding 
these measures. 

Prompting and Project Completion 

As anticipated, students in activity prompt (AP) condition in Study 2 were 
significantly more likely to complete all pieces of the project. In the "All The 
News" project, approximately 78% of the AP group completed all pieces of the 
project, as compared to 32%c of the self-monitoring prompt (SMP) group and 42%c of 
the control group. A Fisher's Exact test found that the AP group was significantly 
more likely than the SMP group (p < 0.002) and than the control group (p < 0.04) to 
complete all the pieces of the project. 

To provide the most direct scaffolding, the AP group was explicitly prompted wdth 
sentence-starters for each piece whereas the other groups received only the 
section headings and some prose discussing each piece. The headings and prose 
were present for each of the three groups. Although the existence of a difference 
in completion rates was not surprising, the degree to which the conditions 
performed differently was unexpected. 

In this critique project, activity prompts provide students with tangible reminders 
to discuss the important aspects of the project. In this way, they help to reduce 
students' cognitive load. These prompts also present opportunities for students to 
identify pieces of evidence or aspects of the article that require improvement and 
provide reasons behind their thinking. They guide the inquiry process and thus 
help students to walk through the activities step-by-step. 
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Prompting and Project Quality 

Although the activity prompts proved more successful at encouraging students to 
complete all the pieces of the project, the activity prompts did not facilitate 
improved quality. Identifying a single indicator of quality across projects is 
inappropriate because the cognitive goals of the two t 5 q)es of projects differ. Based 
on measures appropriate for each project, results indicate that students in the 
self-monitoring conditions tended to create higher-quality projects. 

Justification 

Not surprisingly, the activity prompts in the "Aliens" project were more 
successful at directly eliciting scientific thought than were the more general self- 
monitoring prompts. However, indirect results of the self-monitoring prompts are 
apparent. (See Table 2 for the results of the chi-square tests for Study 1.) The 
students in the condition with self-monitoring prompts were less likely to give 
purely descriptive explanations than were those students in the other group, and 
were significantly more likely to use at least one scientific principle in their 
designs iy} = 8.92, p < 0.05, df = 2). There was no difference in the overall quality of 
the two groups' projects, a phenomenon that is particularly interesting given that 
these prompts were essentially additional cognitive requirements for the students 
who completed them; the classes who received only activity prompts had, in effect, 
more time for their reports. 





Activity Prompt 
Group 

(N = 33) 


Self-Monitoring 
+ Activity 
Prompt Group 

(N = 32) 


Level of Explanation 


(not significant) 


Scientific 


36.36<7f 


25.00% 


Explanatory 


33.33% 


59.38% 


Descriptive 


30.30% 


15.62% 


Number of Principles 


(P < 


0.05) 


More than one 


51.52% 


37.50% 


One 


21.21% 


59.38% 


Zero 


21.21% 


3.12% 


Activity Prompt 


(not significant) 


Scientific Content 


33.33% 


46.88% 


No Scientific Content 


67.67% 


53.12%- 



Table 2: Chi-Square Results for Study 1 



The self-monitoring prompts did not appear to move students to the highest 
possible levels of performance in that they did not elicit explanations involving 
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multiple principles. Instead, the presence of the prompts affected whether 
students gave any explanation or justification. 

It appears that there was also an interaction between the self-monitoring prompts 
and the activity prompts, which simply asked students to write why their design 
would work well. Students who received the self-monitoring prompts were more 
likely to include scientific content in the end-of-design activity prompt. Of the 
students in the self-monitoring prompt condition, 47% included scientific content 
in the activity prompt, as compared to only 33% of the students in the activity 
prompt group. (See Table 2.) Though this difference is not significant, it lends 
further credence to the hypothesis that students who created an explicit plan for 
their activities and reflected on their work were more aware of the need to provide 
scientific justification for their decisions. It also suggests that a mix of prompts 
may prove most beneficial for some students' understanding. Helping students to 
express their scientific ideas facilitates their further understanding of those 
ideas. 

Principled knowledge integration 

Unlike the "Aliens" design project in Study 1, the "All The News" critique project 
tended to elicit a relatively standard response in terms of the number of principles 
considered applicable by most students. The quality measure used in Study 2, 
then, is the degree to which the principles were linked to cites of the students' 
other knowledge (such as labs they had done in class or experiences from their 
daily lives). 

Students in the SMP condition in the "All The News" project were significantly 
more likely than those in the AP condition to exhibit evidence of principled 
knowledge integration. Principled knowledge integration is defined as knowledge 
integration around one or more scientific principles and is indicated by the 
combinations of cites used in the explanations. In particular, the analysis 
investigates whether principles were used in the explanations, and if they were, 
how many other types of appropriate cites there were in the explanations (cf. 
Clark, 1996). Of course students may not express all the cites they are building on; 
the principled knowledge integration measure simply provides our best insight 
into students' integration of their knowledge. 
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The AP group gave significantly more principle-only explanations than the SMP 
group, whereas the SMP group gave significantly more explanations exhibiting 
principled integration = 7.83, df = 3, p < 0.05). That is, the SMP group tended to 
explain phenomena using a principle and one or more other types of cites, such as 
everyday experiences or laboratories done in class. 

Although the difference is reduced to only a trend when the results are combined 
(comparing all explanations exhibiting principled knowledge integration with all 
those not exhibiting it, rather than considering each individual type of 
explanation separately), the results are still striking. (See Figure 2.) The SMP 
group tended to exhibit greater evidence for principled knowledge integration. 

The AP group, on the other hand, tended to either use a single principle (without 
other cites) or no principles. 
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Figure 2: Principled Knowledge Integration 

It appears that self-monitoring prompts encourage students to demonstrate or 
develop an integrated understanding of the science concepts. These prompts may 
foster a "think-through" approach to the project, as compared to the "walk- 
through" approach engendered by the activity prompts. 
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Guidelines written in ATN 



The "All The News" project required students to write to the imaginary reporters 
with guidelines for using evidence. The guidelines were meant to be specific, in 
order to be as helpful to the “repox lers" as possible. Using a Jb'isher Jbixact test, the 
AP group was significantly more likely than the control group to include multiple 
guidelines (p < 0.04). However, the control (beliefs prompts) group made 
significantly more specific guidelines than the AP group (p < 0.03). There is a 
strong trend for the control group to make more specific guidelines than the SMP 
group, as well (p < 0.1). There was no difference between the AP and SMP group 
in the number or specificity of guidelines made. 



Thus, students in AP condition wrote significantly more numerous, but also more 
general, guidelines than did the control group. For example, students in the 
experimental groups might write "Make the evidence true and relevant" while 
students in the control group might write "Be sure to control your variables", 
which gives a specific suggestion for a way to make sure the evidence is "true." 
While both types of responses are valid, the latter would be more useful if one were 
learning to use evidence. 



It appears that the activity prompts were more likely to help students write more 
guidelines, but that their guidelines were not any better. This trend is also, 
however, true of the SMP group. Different explanations may apply to these two 
findings. It may be that students receiving self-monitoring prompts got stuck in a 
the more abstract mode of thinking required by self-monitoring prompts. The 
activity prompts, which influence the "quantity" of students' work in the overall 
project coding, may have simply encouraged a "more is better" approach. 

Major Dimensions of Self Monitoring Prompt Responses 

How might the self-monitoring prompts encourage these different types of 
thinking? To gain understanding of the mechanism at work, students' responses 
to the prompts themselves were reviewed. 

In Study 1, the self-monitoring prompts in the "Aliens" project were very 
successful in their stated goal: more than half of the "plan ahead" prompts 
involved a restatement of the goal of the activity on which they were embarking, 
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and most of the "look back" prompts involved students reflecting on their projects 
and asking valid logistical questions about the procedures involved in completing 
the project. Other reflections included (mostly unelaborated) suggestions for 
improving their designs. 

Some interesting themes fall out of a more comprehensive review of the self- 
monitoring prompt responses in the "All The News" project in Study 2. These 
themes can help us begin to understand how self-monitoring prompts might 
foster an integrated understanding in students. Although not all of the responses 
indicate that students are taking an integrative view of the project, there are 
numerous instances in which this happens. And, importantly, even the more 
detail-oriented responses to the self-monitoring prompts could also help students 
identify, clarify, and understand the goals of the project and the activities. Thus, 
both types of responses are valid and useful to varying degrees. 

Characterizing students' responses to the self-monitoring prompts also helps us 
identify refinements to encourage the most useful kinds of thinking. Clearly 
prompts can be interpreted in different ways (as the following examples will 
amply show). The goal is to develop a program of prompts that is most likely to 
help students to take whatever steps are appropriate for them to move toward an 
integrated understanding. Different programs of prompts maj' benefit different 
students. 

Some of the major dimensions are discussed and exemplified here. Extreme 
responses are presented as examples; however, it is important to remember that 
responses fall in a range along each dimension. (Students' responses are more 
clear when taken within the context of the article they were critiquing. The text of 
the article is included in Appendix D.) 

Elaborated versus unelaborated responses 

Self-monitoring prompts can elicit elaborations. Some students merely "fill in the 
blank", while others give a more complete explanation of their response: 

Pieces of evidence or claims in the article we didn't understand very well included... the 
anti-heat shirt claim, and the greenhouse effect claim. (UN ELABORATED) 

Pieces of evidence or claims in the article we didn't understand very well included... sonic 
of the evidence was unclear and badly represented. Also the chemicals in the anti-heat shirt 
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were not explained and we didn’t really know why it is so great of a shirt. A black oven 
does not attract heat away from the house. (ELABORATED) 

The information we need to include in our critique is stuff like... three pieces of evidence 
and claims they support. (UNELABORATED) 

The information we need to include in our critique is stuff like... what we thought about the 
evidence. Also what was wrong about the claims. We need to include what we thought was 
done well. (ELABORATED) 

The information we need to write in our letter includes... what was scientific and non- 
scientific on the different evidence and claims. EX. The graph was scientific, the opinion 
was not. We will include the basic rule for writing scientific articles. That is to use facts 
not opinions. (ELABORATED) 

Generally, elaborated responses would appear to be more useful in helping 
students develop a deeper, more integrated understanding. 

Analysis of individual ideas versus integration of multiple ideas 

Self-monitoring prompts can encourage knowledge integration, both directly and 
indirectly. Some students discuss individual concepts or pieces of evidence while 
others integrate (or indicate a desire to integrate) their knowledge. Integration 
might involve multiple principles or pieces of evidence, or it might link students' 
real-life experiences with a particular concept under consideration. 

Pieces of evidence or claims in the article we didn't understand very well included... 
thing about how black colors absorbs heat instead of attracting heat. Is there really a 
difference? (ANALYSIS) 

Pieces of evidence or claims in the article we didn't understand very well included... 
the greenhouse effect has to do with solar oven. (INTEGRATION) 

Pieces of evidence or claims in the article we didn't understand very well included... 
do we need to cook outside with that special stove because we don't believe it makes a 
difference, you could use a grill instead. The evidence we didn't understand we wrote in 
our other notes. (INTEGRATION) 

We would have understood the article better if... you told us which chemical the anti-heat 
shirt had. (ANALYSIS) 

We would have understood the article better if... they would explain how the light energy 
matters about heated up the house. Maybe they should give out examples and evidence. 
(INTEGRATION) 

Students reap benefits from both types of analysis. If students identify specific 
science concepts they do not understand, they move one step closer to developing 
an understanding of that concept. However, by taking an integrative approach, 
students may clarify their goals for the project as a whole which may help them to 
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integrate their knowledge. Students may also identify aspects of their 
understanding that could be linked, refined, or reconciled. Integration is 
obviously important, and a deep understanding of individual pieces provides the 
foundation for that integration. 

New attitudes toward learning versus traditional attitudes 

Self-monitoring prompts can facilitate articulation of students' attitudes toward 
learning and working on the project. Some students appear to have a traditional 
view of school, seeing the project as a set of paces to be made. Some similarly put 
the onus of responsibility for their learning on others. Others demonstrate a new 
attitude, adopting a knowledge integration approach, going beyond the 
requirements of the project, and taking personal responsibility for their learning. 

To do a good job on this project, we need to ... look at all the evidence, follow the directions 
given, and do our best to complete every step. (TRADITIONAL) 

The part of critiquing that 's hardest for us is... that we are supposed to have a curious eye 
on every aspect of the article. For us that is new and hard for us and it is hard to cope with 
that. We think that with time and effort we will be better critiquers and have a curious eye 
on everything. (NEW) 

To answer those questions we will... ask Mr. K. (TRADITIONAL) 

To answer those questions we will... look back to our old labs and look at the information 
more carefully and see if there is any new evidence. (NEW) 

Most students encouraged in the traditional mind-set since they started school; it 
is not at all surprising that they often view science as a collection of facts to be 
memorized (Davis, 1995; Linn & Songer, 1993; Songer & Linn, 1991) and science 
projects as yet another opportunity to fill in the blanks. KIE projects attempt to 
foster a change in these attitudes. To whatever extent the students exhibit an 
understanding of these new learning goals, they may also prove more likely to 
work to achieve the integrated understanding encouraged by those goals. While 
"following the directions given" may be a good idea, "having a curious eye" may be 
more likely to facilitate the integration of knowledge. 

Influences on prompt development 

How will these findings influence the program of self-monitoring prompts 
students experience? Since prompts are interpreted by individual students, we 
cannot be certain of developing foolproof "integration" prompts. However, 
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investigating the trends visible in the prompts — and seeing how different students 
understand and respond to them-allows refinement of the prompts to move 
toward that goal. Prompts should help students elaborate on their thinking and 
encourage an integrative stance without eliminating the use of an analytic stance. 
Such self-monitoring prompts may help students see and articulate the new goals 
of science and may help them to achieve an integrated understanding of the 
science concepts under consideration. 

Discussion 

This set of investigations shows that metacognitive prompts have both strengths 
and weaknesses. Activity prompts help students finish activities but do not 
necessarily help the students to develop an integrated understanding. Self- 
monitoring prompts encourage students to plan their activities, remind students 
to reflect on their own understanding, and improve students' likelihood of 
explaining and justifying the decisions they make. These results have directly 
affected the design of the KIE software and projects. However, merely identifying 
the effects of the two types of prompts is insufficient to fully improve students' 
learning conditions. We must also investigate the cognitive mechanisms by 
which the prompts work. The two research questions framing this research 
addressed both of these issues, and will be reviewed now. 

First, prompts do have specific effects. The results reported here indicate that 
while activity prompts guide students to complete the important pieces of a 
project, self-monitoring prompts encourage them to link those pieces and 
integrate their knowledge. Both prompt types help some students more than 
others. And, findings from Study 1 indicate that the two types of prompts may 
have a synergistic effect, as well. 

But how can we explain these instructional effects? Activity prompts may lessen 
the cognitive load on students, helping them to identify how to accomplish the 
steps necessary for an activity and to remember to do each of those steps in the 
first place. These prompts may, however, encourage a piecemeal view of the 
process, by emphasizing each step rather than how the steps work together to 
achieve a particular goal. The students may have an integrated understanding of 
the concepts, but if that is the case they do not demonstrate it. 

Metacognitive Scaffolding... 

Elizabeth A. Davis o , 

page 19 “ 



On the other hand, self-monitoring prompts help students integrate their 
knowledge around principles or demonstrate such an understanding. Since the 
self-monitoring prompts used in the studies do not directly elicit integrated 
knowledge, it appears that students simply need scaffolding to think about their 
goals for a project and their progress on that project, topics on which students are 
not generally requested to think. Perhaps as a result, they are better able to 
integrate their understanding. In fact, some students indicate in their responses 
to self-monitoring prompts that they are aware that the KIE projects require them 
to embark on a different, more integrative type of thinking, as indicated by the 
students who are struggling to develop a "curious eye." It may be that self- 
monitoring prompts help students view KIE projects differently from how they 
typically view school projects — a "think-through" approach — and thus engage in 
the task in an inherently different way. 

The implied synergistic effect may indicate that at least some students benefit 
from, both aspects of metacognitive prompting. Students may attain an improved 
understanding of their goals through "planning ahead" prompts, make 
appropriate steps toward those goals through activity prompts, and improve their 
understanding and their work through "looking back" prompts. 

Further work is necessary to better understand the effects of the two types of 
metacognitive prompts, in particular the possible synergy. One last large question 
must be addressed, as well: What influences the benefits students reap from 
different types of prompts? I hypothesize that different students will benefit from 
different prompts or mixes of prompts, based in part on their beliefs about science 
and about learning science. Those beliefs may influence how students go about 
doing science, which in turn would influence their use of prompts. 

Specifically, based on the results discussed here and other related research, I 
hypothesize that activity prompts are necessary for some students who would not 
otherwise complete the process required by the project, but that self-monitoring 
prompts may help some students (who may already be proficient at the process 
required) look at the bigger picture. I further hypothesize that self-monitoring 
prompts may benefit some students who do not exhibit proficiency at or 
understanding of the process of science, by encouraging them to clarify the overall 
goals of each activity on which they are embarking and reflecting on how each 
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activity moves them toward completion of the final goal. I also hypothesize that a 
mix of activity prompts and self-monitoring prompts may be most beneficial to 
some students, who need a bit of the process guidance provided by the activity 
prompts but have the ability to reflect on and integrate their understanding, as 
encouraged by the self-monitoring prompts. It may be that this mix is beneficial 
for a majority of the students. Last, I h}TDothesize that some students do not need 
either type of prompts. These students may find self-monitoring prompts 
irrelevant (because they already know how to manage a large project) and activity 
prompts stifling (because they already know how to take the necessary steps). 

Both types of prompts can interrupt independent reasoning and distract students 
who already understand what steps to take. Contextualized prompting, while 
often helpful, thus requires judicious use of prompts. Studies are underway to 
further investigate the effects of metacognitive prompts. In particular, work is 
ongoing in the identification of relationships between students' responses to the 
self-monitoring prompts and the work they do on the projects, the relationship 
between completion and project quality within individuals, and students' views of 
the purposes of the prompts themselves. Results of this work will further our 
understanding of how to foster metacognitive activity for individual students. 

The research described here, in conjunction with the work in progress, will 
improve KIE students' learning opportunities by improving both the curriculum 
materials (in particular, the prompts) and the software itself (the cognitive- 
guidance system). Through careful analysis of how students respond to 
particular types of prompts, we can develop curricula suited to their particular 
needs. Students who would benefit most from a particular type of prompt (or a 
mix of prompts) would receive that program. Students may also benefit from 
being challenged to experience prompts with which they may not be comfortable, 
to help them develop new ways of thinking. 

The research may also affect the design of instruction by educators not associated 
with KIE. Understanding the benefits of the two t}qDes of metacognitive prompts 
described here may encourage instruction using activity-like prompts and self- 
monitoring-like prompts. Some students may be benefited by being explicitly 
prompted for metacognitive activity. 
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Appendix A: An "Aliens" Document 



Design Dwelling 

by Student-1 & Student-2 
inE period: 



Planning Aiiead: In thinking about our doing our design, we need to think 
about... 



Your assignment is to use the evidence you've seen to design a dwelling for the 
aliens to make their visit on Zumtar as enjoyable and comfortable as possible. 

You need to think of suggestions and designs which would be most helpful in 
maintaining the aliens' ideal temperature throughout the day and night (don't 
forget about the air blast that happens every morning and evening). 

Your design should include what type of material you choose for the walls of the 
dwelling, recommendations for paint colors, and information on the temperature 
of the air blast given. All recommendations should be backed up with scientific 
evidence or thoughts. The same dwelling must work for ALL aliens (you can, 
however, customize different aliens' apartments through the air blast you give 
them). 

Don't forget to check with Mildred for help in working on this! 



Dwelling Design: 

write dwelling design here 

when you're done with the dwelling design; 

My design will be good for the different kinds of aliens because... 

Looking Back: Our design could be better if we... 
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Appendix B: An "All The News” Document 

(Activity Prompt Condition) 

Letter to the Newspaper 

by Student- 1 & Student-2 
KIIE period: 



Your task now is to write a letter to the newspaper on how the article could be 
improved. 

The letter should talk about how each piece of evidence you critiqued helped (or 
didn't help) the claims the article was making, and how the claims help the 
overall argument. If a piece of evidence should be taken out, or if a claim in the 
article needs to be rephrased, explain why. Remember, the Weekly World Science 
News is trying to get their reporters writing more believable and more scientific 
articles. Your summary should provide a set of guidelines for the reporters to 
follow in the future. 

Don't forget to check with Mildred for help in working on this! Mildred includes 
hints on what could be included in each section as well as things to remember 
when you're making suggestions to the people at the newspaper. 

The headings and prompts here are meant to be suggestions. You should use 
your own style to write the report! 



Dear Weekly World Science News, 

Section 1: Introduction 

[Thank you for the opportunity to help the Weekly World Science News. We have 
some suggestions that will make your paper's article much more believable and 
scientific. We will try to explain our suggestions really well so the reporters will be 
able to learn what they should do in the future.] 



Section 2: The Evidence, the Claims, and the Overall Argument 

[Here, we'll present each of three pieces of evidence in turn, explaining why each 
helps (or doesn't help) the reporters make the claims in the article. And, we'll say 
why each piece of evidence or claim should (or shouldn't) be included in the 
article.] 

[Evidence-1...] 

[Claim-1...] 2\' 
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[Evidence-2...] 

[Claim-2...] 

[Evidence-3...] 

[Claim-3...] 

[Now, we'll talk about the overall argument, and how the claims and evidence 
support or don't support the argument of the article.] 

[The argument...] 

[Readers would get more from the article if the article...] 

Section 3: Evidence Guidelines for the Reporters 

[A^ou->, we'll give you a set of guidelines that your reporters should follow when 
they use evidence.] 

[When you think about evidence, in general you should make sure to consider...] 
[When you write an article based on evidence, the article should...] 



Section 4: Conclusion 

[We hope our report on your paper's article is helpful. We tried to provide 
information on which evidence and claims are scientifically OK and which are 
bogus. We also tried to give you reasons for every decision or suggestion we made. 
We tried to keep reporters in mind as we wrote our reports, so it would be 
appropriate for you. We tried to make our suggestions and comments very clear .] 

Sincerely, 

[your names] 
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Appendix C: An "All The News" Document 

(Self-Monitoring Prompt Condition) 
Letter to the Newspaper 
by Student- 1 & Student-2 
KIE period: 



Planning Ahead: 

Thinking Ahead: Specific things we need to think about as we write our letter 
include... 

What to Include: The information we need to write in our letter includes... 



Writing the Letter: 

Your task now is to write a letter to the newspaper on how the article could be 
improved. 

The letter should talk about how each piece of evidence you critiqued helped (or 
didn't help) the claims the article was making, and how the claims help the 
overall argument. If a piece of evidence should be taken out, or if a claim in the 
article needs to be rephrased, explain why. Remember, the Weekly World Science 
News is trying to get their reporters writing more believable and more scientific 
articles. Your summary should provide a set of guidelines for the reporters to 
follow’ in the future. 

Don't forget to check with Mildred for help in working on this! Mildred includes 
hints on what could be included in each section as well as things to remember 
when you're making suggestions to the people at the newspaper. 

The headings and prompts here are meant to be suggestions. You should use 
your own style to write the report! 



Dear Weekly World Science News, 

Section 1: Introduction 

[Thank you for the opportunity to help the Weekly World Science News. We have 
some suggestions that will make your paper's article much more believable and 
scientific. We will try to explain our suggestions really well so the reporters will be 
able to learn what they should do in the future.] 
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Section 2: The Evidence, the Claims, and the Overall Argument 



[Here, we'll present each of three pieces of evidence in turn, explaining why each 
helps (or doesn't help) the reporters make the claims in the article. And, we'll say 
why each piece of evidence or claim should (or shouldn't) be included in the 
article.] 



[Now, we'll talk about the overall argument, and how the claims and evidence 
support or don't support the argument of the article.] 



Section 3: Evidence Guidelines for the Reporters 

[Now, we'll give you a set of guidelines that your reporters should follow when 
they use evidence.] 



Section 4: Conclusion 

[We hope our report on your paper's article is helpful. We tried to provide 
information on which evidence and claims are scientifically OK and which are 
bogus. We also tried to give you reasons for every decision or suggestion we made. 
We tried to keep reporters in mind as we wrote our reports, so it would be 
appropriate for you. We tried to make our suggestions and comments very clear.] 

Sincerely, 

[your names] 



Looking Back 

Checking Our Understanding: Questions we still have are... 

Improving Our Understanding: To answer those questions, we will... 

Checking Our Work: In looking back at what the editor wanted, we think she 
will like our letter because... 

Checking Our Work: The editor might not like... 

How We Spent Our Time: As we worked on this project, we wish we'd spent 
more (or less) time on... 



iU) 
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Appendix D: The "All The News" Article 
Keeping Your Cool in the Summertime 
by Rocky Reporter and Jana Journalist 



Summer is a time for fun and sun, but when it gets too hot, watch out! We've 
figured out how to beat the heat this summer. To keep your house cool this 
summer, do your cooking outside. Outdoor cooking attracts heat away from the 
house and keeps everyone cool. The new solar ovens are the best, using up lots of 
heat that would otherwise heat houses. 

Claim 1: Energy conversion principles indicate that black attracts heat 

When made correctly, solar ovens use energy conversion principles to put heat to 
good uses. Solar ovens often have black lids to make them heat up faster. We asked 
local residents why the black lids worked. One woman interviewed reports, "I've 
seen it happen myself. When my friend Bill and I went to the beach. Bill stayed 
cool longer in his white T-shirt than I did in my black dress." Obviously, black 
items attract the heat better than white items. 

Claim 2: Heat sources cause the temperature to go up 

Why move the ovens outside? Everyone knows that anything that gives off heat will 
heat up the space it's in. So since ovens heat up kitchens , moving them outside 
will cool the house down. This works best in houses with small kitchens since 
studies show that heat sources heat up small rooms faster than large rooms. 

Claim 3: Ovens and other objects use up sunlight, thereby lowering 
temperatures 

The third point, then, is that having a solar oven outside would keep your house 
cool by using up some of the sunlight that would otherwise have entered your 
house and made it hot. Students performing an experiment on the greenhouse 
effect determined that the heat can be trapped in containers where light can get in 
easily but heat can't get out. We've experienced the same phenomenon when we 
left our cars in the sun with the windows shut; the heat built up qmckly. If 
possible, leave the windows open and the shades drawn so less sunlight comes in. 
The solar oven uses up heat in a way similar to how the new "anti-heat" shirt 
does. The shirt reportedly keeps you cool because it is dipped in a special chemical. 
The manufacturers say that this chemical can keep you cool even in the hottest 
weather. 

You can see, then, that the answer to the question "How can I stay cool in the 
summertime?" is clear. Just cook with a solar oven and use other items to use up 
the sunlight, and you'll stay nice and cool while everyone else is sweating. 
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